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Abstract 

Cost  growth  continues  to  be  a  serious  concern  in  major  acquisition  programs.  A 
variety  of  causes  have  been  identified  for  cost  growth,  including  low  initial  cost 
estimates,  complex  acquisition  processes,  and  immature  technologies.  Incentive- 
based  systems  have  been  employed  in  an  attempt  at  cost  savings,  with  mixed 
results  at  best.  This  paper  examines  the  role  of  process  and  incentive 
characteristics  in  cost  growth.  In  particular,  we  study  process  concurrency,  types  of 
incentive  contracts  employed  and  the  transfer  point  from  cost-plus  to  fixed-price 
contracts,  and  the  resulting  effects  on  cost  growth  in  the  F-35  Joint  Strike  Fighter 
program.  The  F-35  program  currently  is  in  low-rate  initial  production.  The  emerging 
paradigm  of  organizational  simulation  is  used  in  this  study,  since  it  combines  process 
representations  to  model  acquisition  processes  and  agent  representations  to  model 
multi-actor  behavior,  including  reaction  to  incentives.  Simulation  experiments  are 
conducted  and  analyzed  to  determine  the  effects  of  the  factors  described  above  on 
cost  growth. 
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Introduction 

Cost  growth  has  been  a  significant  problem  in  major  DoD  acquisition  programs.  A 
recent  report  from  the  Government  Accountability  Office  (GAO)  notes  that,  for  the  fiscal  year 
2008  portfolio  of  weapons  systems,  there  has  been  cost  growth  of  $296  billion  (GAO,  2009). 
In  addition,  since  2008,  GAO  (2011)  notes  that  there  has  been  $135  billion  in  total  cost 
growth,  $70  billion  of  which  cannot  be  explained  by  changes  in  quantities  ordered.  Cost 
growth  can  result  in  fewer  systems  being  produced  than  envisioned  or  desired  (e.g.,  F-22), 
or  in  program  cancellation  (e.g.,  Navy  Area  Missile  Defense).  In  the  current  and  projected 
fiscal  environment,  there  is  considerable  pressure  to  rein  in  cost  growth. 

Cost  growth  is  a  complex  phenomenon  involving  technical  issues,  decision-making, 
contractor  performance,  and  uncertainty.  Therefore,  it  is  not  easily  addressed  or  sometimes 
even  properly  understood.  One  way  to  study  systems  exhibiting  uncertainty  is  computer 
simulation,  in  which  a  model  of  the  system  to  be  studied  is  specified  and  then  analyzed  to 
determine  the  system’s  performance  with  respect  to  different  criteria  under  different 
conditions.  In  computer  simulation,  experiments  can  be  conducted  without  using  the  real 
system,  which  is  advantageous  for  many  types  of  systems  that  simply  would  not  be  used  for 
experimentation.  Here,  we  are  interested  in  studying  acquisition  processes,  which 
encompass  technical  issues  and  technical  decision-making,  and  incentives,  which  affect 
contractor  performance  and  decision-making. 

The  remainder  of  this  paper  is  organized  as  follows.  The  Acquisition  Processes, 
Incentives,  and  Cost  Growth  section  discusses  the  issues  involved  in  cost  growth  from  a 
process  and  incentive  perspective.  The  Model  Description  section  introduces  and  describes 
a  model  of  the  acquisition  enterprise  using  organizational  simulation,  a  relatively  new 
paradigm  for  simulating  enterprise  systems.  The  Experimental  Example  section  presents 
some  simulation  results  illustrating  the  model  concepts.  The  final  section  concludes  with  a 
discussion  on  future  research. 

Acquisition  Processes,  Incentives,  and  Cost  Growth 

Cost  growth  occurs  for  a  variety  of  reasons,  including  uncertainty  and  lack  of 
knowledge  about  technology,  design,  and  manufacturing  (GAO,  2009a).  Candreva  (2009) 
points  to  the  role  of  institutional  factors  in  organizational  failures  such  as  cost  growth.  Our 
previous  research  has  addressed  cost  by  focusing  on  the  process  aspects  of  acquisition. 

For  instance,  we  have  demonstrated  that  evolutionary  acquisition  processes  can  yield 
quicker  deployment  of  capability  than  traditional  acquisition  processes,  but  at  potentially 
higher  cost  due  to  overhead  from  the  increased  frequency  of  development  cycles  (Pennock 
&  Rouse,  2008).  System  modularity  tends  to  reduce  the  overall  life  cycle  cost  when 
sustainment  is  considered  and  can  mitigate  higher  costs  associated  with  larger  production 
levels,  thus  reducing  the  effect  of  cost  growth  (Bodner,  Rahman,  &  Rouse,  2010).  However, 
such  process  modeling  does  not  capture  the  effect  of  incentives,  which  can  be  an  effective 
approach  to  achieving  contractor  performance,  if  properly  applied  (Tremaine,  2008). 

Incentives  and  contract  structures  potentially  play  an  important  role  in  cost  control. 
The  two  main  types  of  contract  structures  are  cost-plus,  in  which  the  government  reimburses 
the  contractor  for  costs  incurred  and  pays  an  additional  amount  for  profit,  and  fixed-price,  in 
which  the  government  pays  a  fixed  price  for  a  set  deliverable  (e.g.,  number  of  systems). 
Typically,  the  particular  type  of  contract  structure  is  used  based  on  the  risk  profile  of  the 
program.  Programs  associated  with  high  levels  of  research  and  development  tend  to  use 
cost-plus  contracts,  since  research  and  development  entail  significant  risk  for  the  contractor. 
Cost-plus  contracts  shift  that  risk  to  the  government.  Fixed-price  contracts,  on  the  other 
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hand,  are  used  for  production,  in  which  costs  are  more  certain.  Fixed-price  contracts  shift 
risk  to  the  contractor. 

Incentives  traditionally  have  been  implemented  via  such  contract  mechanisms  as 
cost-plus-incentive-fee  (CPIF)  or  cost-plus-award-fee  (CPAF).  An  incentive  fee  rewards 
cost  control,  while  an  award  fee  rewards  performance  related  to  non-cost  outcomes. 
Outcomes  may  relate  to  system  capabilities  (e.g.,  speed,  altitude)  or  targets  over  time,  such 
as  the  emerging  concept  of  performance-based  logistics  (Kratz  &  Buckingham,  2009). 

Award  fees  may  be  structured  to  occur  within  certain  evaluation  periods,  and  a  rollover  may 
be  used  to  transfer  an  unearned  fee  to  a  future  period  in  which  it  can  be  earned. 

Competition,  as  opposed  to  non-competition,  can  also  be  considered  as  an  incentive  for  a 
contractor  (Birkler  et  al.,  2001),  although  use  of  competition  is  constrained  in  areas  such  as 
aircraft  acquisition,  due  to  industry  consolidation  (Birkler  et  al.,  2003).  An  award  fee 
structure  for  spiral  development  of  software-intensive  systems  is  presented  by  Reifer  and 
Boehm  (2006). 

The  fundamental  idea  is  to  tie  the  incentive  to  the  desired  outcomes.  This  has  not 
always  worked  in  practice,  however.  A  recent  GAO  report  finds  that  the  current  DoD 
practice  of  using  award  and  incentive  fees  is  ineffective  (GAO,  2005).  Award  fees  often  are 
provided  despite  the  contractor's  not  having  met  performance  criteria,  and  incentive  fees 
have  not  been  shown  to  motivate  contractors  to  control  costs.  In  addition,  DoD  does  not 
have  a  system  to  allow  sharing  of  case  studies  demonstrating  successful  use  of  award  or 
incentive  fees. 

Recent  studies  have  addressed  increasing  the  effectiveness  of  award  fees  and 
incentive  fees.  Using  an  analytic  approach,  Hildebrandt  (2009)  develops  guidelines  for 
effective  decision-making  and  information  availability  structures  applied  to  incentive 
contracts.  Under  some  arrangements,  the  information  required  may  be  demanding.  Four 
programs  are  examined  by  Gilbreth  and  Hubbard  (2008)  to  develop  recommendations  for 
effective  incentive  use.  These  recommendations  include  adequate  training  and  feedback, 
plus  several  specific  recommendations  for  award  fees  (using  a  base  fee,  setting  the  award 
fee  based  on  outcomes  rather  than  time,  relating  the  award  fee  to  the  outcomes  achieved, 
and  using  rollovers  judiciously).  These  results  are  confirmed  and  extended  by  a 
continuation  of  the  study  that  examined  25  programs  (Tremaine,  2008). 

Consider  the  F-35  program,  which  is  developing  and  producing  three  variants  of  a 
next-generation  tactical  fighter  for  three  Service  applications  (Air  Force,  Navy,  and  Marines). 
This  program  currently  is  in  low-rate  initial  production  (LRIP).  The  program  has  seen 
significant  cost  growth  and  schedule  slippage  (GAO,  2009b).  There  are  a  variety  of  causes 
cited,  including  immature  technologies  in  development,  ongoing  design  changes,  inefficient 
production  processes,  and  incomplete  testing  (GAO,  201 1 ).  Concerns  due  to  the  cost  of  the 
program  have  caused  the  National  Commission  on  Fiscal  Responsibility  and  Reform  to 
recommend  significant  reductions  in  quantities  to  be  procured  (NCFRR,  2010)  and  have 
also  prompted  discussions  of  cancelling  the  Marine  variant  (short  takeoff  and  landing). 
Nevertheless,  the  program  has  recently  entered  into  its  fourth  phase  of  LRIP  with  a  switch 
from  a  cost-plus  contract  to  a  fixed-price  contract. 

In  addition,  the  F-35  program  is  of  interest  due  to  the  highly  distributed  nature  of  the 
design  and  production  network  of  contractors.  In  previous  aircraft  programs  (e.g.,  F-16),  the 
aircraft  was  largely  designed  and  built  by  the  prime  contractor,  with  sourcing  of  somewhat 
simple  components  from  suppliers.  With  the  F-35,  major  subsystems  are  sourced  for  design 
and  production  to  other  contractors,  many  of  which  are  international  (Kapstein,  2004). 

Hence,  the  problem  of  government’s  incentivizing  the  contractor  really  becomes  more 
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complex,  as  the  prime  contractor,  in  the  role  of  lead  systems  integrator  (LSI),  must  in  turn 
incentivize  partners  and  other  contractors. 

The  intent  here  is  not  specifically  to  study  the  F-35  program  but  rather  to  study  the 
interaction  of  process  and  incentive  issues  raised  by  the  program  in  the  context  of  cost 
growth. 

Model  Description 

This  research  uses  the  emerging  paradigm  of  organizational  simulation  (Rouse  & 
Boff,  2005)  to  study  the  problem  of  acquisition  cost  growth.  Traditional  simulations  used  to 
study  organizations  can  be  divided  into  three  major  paradigms:  discrete-event  (Law  & 

Kelton,  2000),  system  dynamics  (Sterman,  2000),  and  agent-based  (Hillebrand  &  Stender, 
1994).  Discrete-event  simulation  tends  to  emphasize  the  transactional  nature  of  process- 
based  systems.  System  dynamics,  on  the  other  hand,  represents  continual  accumulation 
processes  affected  by  feedback  flows  and  lags.  Both  are  suitable  for  studying  process- 
based  systems,  depending  on  the  particular  level  of  model  resolution  and  focus.  Agent- 
based  simulation  is  relatively  new  and  emphasizes  the  interaction  of  actors  in  a  system. 
Thus,  it  has  seen  significant  use  in  social  science  research  applications.  Of  interest  here  is 
its  application  to  economic  behavior. 

Organizational  simulation  uses  elements  of  these  three  paradigms  to  model  process 
and  actor  behaviors  in  organizational  systems.  It  has  been  used  in  several  domains  thus 
far,  including  research  and  development  investment  (Bodner  &  Rouse,  2007),  health  care 
delivery  (Rouse  &  Bodner,  2009),  and  computer  server  design  and  development  (Bodner, 
Mutnury,  Cases,  &  Rouse,  2009). 

Process  Model 

Per  DoD  acquisition  policy,  acquisition  is  divided  into  a  number  of  phases  whereby  a 
program  evolves  from  concept  to  deployed  systems.  These  phases  consist  of  concept 
refinement,  technology  development,  system  development  and  demonstration,  production 
and  deployment,  and  operations  and  support.  Various  milestones  and  reviews  exist  in  the 
process  and  serve  as  gates  through  which  the  program  must  have  made  sufficient  technical 
progress  to  pass.  As  the  program  progresses,  costs  are  incurred  and  are  monitored  against 
estimated  costs.  In  addition,  at  various  points  in  the  process,  contracts  are  awarded  that 
cover  specific  deliverables  relative  to  an  acquisition  phase.  At  these  points,  the  amount  of 
the  contract,  the  deliverables,  and  the  structure  of  the  contract  are  in  play.  The  process 
model  is  shown  in  Figure  1 . 


Possible  contract  renegotiations 


Figure  1.  Acquisition  Processes 
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Note  that  sometimes  concurrency  occurs,  especially  between  system  testing  (in 
system  development  &  demonstration)  and  low-rate  initial  production  (in  production  and 
deployment).  While  this  can  expedite  system  delivery,  it  also  entails  risks,  since 
manufacturing  an  unproven  system  may  entail  redesign  and  rework  if  undiscovered  flaws 
exist.  This  risk  relates  directly  to  potential  cost  growth. 

Actor  Model 

The  actor  model  consists  of  the  set  of  actors,  their  behaviors,  and  their  interactions 
with  one  another.  The  government  is  one  actor,  and  then  contractors  are  modeled  as 
separate  actors. 

Given  that  we  are  interested  in  incentives,  we  use  the  principal-agent  framework  as 
the  basis  for  modeling  actor  interactions.  The  principal-agent  framework  is  used  in  micro¬ 
economics  to  model  the  interaction  of  two  actors,  one  of  which  (the  principal)  utilizes 
another  (the  agent)  to  perform  a  task  (Kreps,  1990).  This  framework  introduces  a  number  of 
problems  such  as  moral  hazard  and  information  asymmetry.  Typically,  the  principal  must 
design  a  contract  mechanism  that  works  to  motivate  the  agent  and  hopefully  addresses  any 
problem  situations. 

In  the  case  of  acquisition,  the  government  serves  as  the  principal  and  the  prime 
contractor  as  its  agent.  In  turn,  other  contractors  may  be  agents  to  the  prime  contractor’s 
role  as  principal.  In  a  complex  supply  network,  the  principal-agent  representation  can  be 
used  to  model  the  many  different  tiers  of  contractors  that  exist,  as  shown  in  Figure  2.  Note 
that  a  contractor  can  be  an  agent  to  multiple  other  contractors.  Sometimes  this  is  known  to 
the  principals;  other  times,  it  is  not  known.  In  addition,  a  contractor  may  be  an  agent  to 
multiple  contractors  located  in  different  programs  (i.e.,  different  supply  networks  under 
different  prime  contractors,  who  may  be  competitors).  Thus,  the  principal-agent  relations  in 
a  real  supply  network  can  be  quite  complex.  Such  complexity  is  usually  not  solvable  by 
analytic  models,  leaving  simulation  as  an  important  method  in  their  study. 


G  =  Government 
P  =  Prime  contractor 
C  =  Contractor 


Figure  2.  Actor  Network 


Incentive  Model 

Here,  we  describe  a  simplified  incentive  model  that  can  serve  as  the  basis  for  a 
principal-agent  model  of  acquisition.  Assume  that  the  government  is  the  principal  and  that 
the  prime  contractor  is  its  agent.  The  agent  has  a  utility  function  representing  value 
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received  from  working  for  the  principal.  We  assume  a  functional  form  as  shown  in  Equation 
1 ,  where  U  represents  the  utility,  w  represents  the  contractual  payment  from  principal  to 
agent  (with  diminishing  marginal  returns),  and  a  represents  the  effort  expended  by  the 
agent.  In  the  case  where  there  is  no  contract,  the  agent  maintains  a  reservation  utility  Ur. 

lawmaigW  (i) 

To  simplify,  we  assume  that  the  agent  can  expend  a  high  level  of  effort  ah  or  a  low 
level  of  effort  a/  during  the  contractual  term.  Obviously,  ah  >  a/.  Now  suppose  that  the 
principal  and  agent  enter  into  a  performance-based  contract  whereby  the  agent  is  paid 
according  to  its  performance  during  the  period.  Again,  for  the  sake  of  simplicity,  we  assume 
three  levels  of  performance  (low,  medium,  and  high).  Payment  is  as  follows. 

■  The  agent  is  paid  x02  if  performance  is  low. 

■  The  agent  is  paid  x12  if  performance  is  medium. 

■  The  agent  is  paid  x22  if  performance  is  high. 


Obviously,  x0  <  xi  <  x2.  The  performance  is  uncertain,  but  it  is  based  in  part  on  the 
level  of  effort  expended  by  the  agent.  Let  P(a)  be  the  performance  level  achieved  by  the 
agent  as  a  function  of  effort,  and  assume  that  there  are  three  levels  of  performance — P0 
(low),  P-\  (medium),  and  P2  (high)  — with  P0  <  Pi  <  P2.  We  model  the  performance  according 
to  the  following  functional  form. 


The  terms  p?  and  p?  represent,  respectively,  probabilities  associated  with  mapping 
the  agent’s  effort  (low  or  high)  to  performance  outcomes.  Even  if  the  agent  expends  high 
effort,  there  is  a  probability  of  low  or  medium  performance.  We  assume,  however,  that  low 
effort  is  more  highly  correlated  with  low  performance  than  with  high  performance  and  vice 
versa.  Thus,  we  assume  that  >  p{  >  and  p*' 


With  regard  to  the  contractual  agreement,  there  are  three  scenarios.  The  agent  can 
decline  the  contract,  the  agent  can  accept  the  contract  and  expend  low  effort,  or  the  agent 
can  accept  the  contract  and  expend  high  effort.  In  the  former  case,  the  agent’s  utility  is 
simply  the  reservation  utility  Ur.  In  the  latter  two  cases,  the  agent’s  expected  utility  is  given 
in  Equation  4. 

(4) 

The  contract  can  be  viewed  from  the  perspective  of  either  the  principal  or  the  agent. 
The  principal,  of  course,  wants  the  agent  to  expend  maximum  effort  so  as  to  maximize  the 
chance  of  high  performance.  On  the  other  hand,  the  principal  does  not  want  to  overpay  the 
agent.  Thus,  one  way  in  which  the  principal’s  objective  can  be  stated  is  to  minimize  the 
payment  subject  to  the  condition  that  the  agent  expends  high  effort. 


(5) 
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From  the  agent’s  perspective,  the  only  value  from  expending  high  effort  occurs  when 
the  expected  utility  from  high  effort  is  greater  than  both  the  reservation  utility  and  the 
expected  utility  of  expending  low  effort.  That  is, 


zmmimf. 


(6) 


&23HBB2S&  (?) 

Combining  Equations  5-7  yields  a  constrained  optimization  problem,  which  is 
solvable  if  the  values  for  parameters  are  known  and  fixed. 

Cost-Plus  Model 

As  the  agent  works  on  behalf  of  the  principal,  it  incurs  costs  for  effort.  We  assume, 
in  general,  that  the  cost  incurred  at  over  time  horizon  t  during  a  particular  acquisition  phase 
has  a  fixed  set-up  cost  a0  and  then  is  linear  in  time  with  respect  to  a  constant  burn  rate  m, 
where  m  is  dependent  on  the  level  of  effort  and  where  t  is  measured  from  the  beginning  of 
the  phase. 

SSniME  (8) 

From  a  cost-plus  perspective,  for  the  agent  to  be  interested  in  the  contract,  payment 
from  the  principal  must  cover  the  agent’s  cost  and  then  provide  a  profit.  Moreover,  the  utility 
from  the  principal’s  payment  and  the  agent’s  cost  (Equation  1)  must  be  greater  than  the 
agent’s  reservation  utility. 

In  cost-plus  arrangements,  the  duration  of  the  acquisition  phase  can  be  estimated 
but  often  varies  from  the  estimate,  driving  cost  growth.  Ideally,  the  principal  desires  that  the 
agent  expend  high  effort,  expecting  performance  to  be  high.  From  this,  the  principal  can 
estimate  the  time  remaining  in  an  acquisition  phase,  as  well  as  a  cost. 

Let  T  be  the  total  completion  time  needed  for  an  acquisition  phase.  Let  ^  be  the 
initial  estimate  for  time  remaining  (i.e.,  is  an  estimate  for  7),  and  let  T£  be  the  remaining 
left  for  completion  at  time  t  (i.e.,  once  the  phase  has  commenced).  At  the  end  of  each  year 
t,  the  time  remaining  can  be  updated  as  follows,  where  the  agent’s  performance  is 
measured  in  time  progression  toward  completion. 

mmam.  <»> 

Note  that  the  phase  would  end  when  ^reaches  zero  for  some  f,  at  which  time  T 
becomes  known.  Assuming  that  the  principal  pays  the  agent  annually  for  the  contract 
duration,  the  principal’s  cost  ct  in  year  t  can  be  computed. 


Let  Cj  be  the  total  cost  to  the  principal  for  the  acquisition  phase,  and  let  ££  be  the 
estimate  for  CT  at  time  t.  Letting  Ct  be  the  cost  incurred  by  the  end  of  year  t,  total  costs  can 
be  tracked  and  the  estimated  phase  cost  can  be  updated. 

(ii) 


(12) 
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Letting  Cg  be  the  initial  estimate  of  costs  to  be  incurred  by  year  t  (t  =  1 , 2,.. 
allows  incremental  cost  growth  to  be  measured  as  and  total  cost  growth  to  be 

estimated  as  £ i  —  tf*. 


Fixed-Price  Model 


Suppose  that  the  principal  uses  a  fixed-price  contract  for  the  agent’s  work.  We 
assume  a  firm  fixed-price  contract,  rather  than  an  instrument  that  allows  some  cost  to  be 
transferred  from  the  agent  to  the  principal.  Hence,  cost  growth  is  not  an  issue  for  the 
principal.  This  situation  typically  occurs  in  production,  where  there  is  less  outcome  risk. 
Here,  economies  of  scale  are  usually  present.  In  micro-economics,  economies  of  scale  are 
modeled  using  a  Cobb-Douglas  production  function  (Kreps,  1990).  Let  X  be  the  amount  of 
input  resources,  in  terms  of  labor,  capital,  and  materials;  N  be  the  output  in  terms  of  number 
of  units  produced;  and  b  be  the  scale  factor.  The  fundamental  relation  between  input  and 
output  is  shown  in  Equation  13. 


•Ml  Ei 


If  £)  >  0,  then  there  are  increasing  returns  to  scale,  meaning  that  the  production  is 
more  efficient  as  more  units  are  produced.  This  discussion  assumes  application  in 
production,  although  the  model  may  be  extended  to  other  phases.  Assuming  a  constant 
per-unit  cost  of  input  S,  the  total  cost  of  the  production  phase  can  be  modeled  as  follows: 

nuum  o-*> 


The  term  is  defined  as  the  production  efficiency.  The  production  efficiency 

is  the  performance  measure  used  for  the  agent,  and  the  unit  cost  of  input  is  the  level  of 
effort.  Similar  to  the  cost  model,  we  use  three  different  levels — a0  (low),  a- \  (medium),  and 
a2  (high) — with  a0  <  «i  <  a2  (lower  a  implies  better  efficiency).  Similar  to  the  cost  model,  the 
effort  has  two  levels:  6/  (low  effort)  and  Bh  (high  effort).  A  higher  effort  may  involve,  for 
example,  more  investment  per  unit  of  labor  in  training  or  per  production  line  in  precision 
machinery,  with  the  intent  to  achieve  better  downstream  efficiency.  Thus,  Bh  >  B\.  Note  that 
there  may  be  a  fixed  cost  associated  with  effort  that  is  not  modeled  here.  Equations  15  and 
16  relate  performance  to  effort  in  a  probabilistic  framework  as  before,  with  'PL  ar|d 


i>V'J 


(15) 


(16) 


If  the  agent  chooses  to  enter  into  a  fixed-price  contract  with  the  principal,  where  Y  is 
the  per-unit  output  price,  the  agent  then  must  choose  what  level  of  effort  to  expend  based 
on  the  expected  gain  G(B),  which  is  a  function  of  payment  from  the  principal  and  costs 
associated  with  production  and  with  effort. 


(17) 
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is  maximized.  However,  this 


The  agent  selects  the  level  of  effort  whereby 
formulation  does  not  address  two  of  the  principal’s  concerns  regarding  the  contract — the 
selection  of  an  appropriate  value  of  Y  and  the  timely  delivery  of  output  units.  The 
performance  measure  a  is  primarily  of  interest  to  the  agent  unless  it  can  be  translated  into 
schedule  performance.  Two  methods  of  doing  this  are  using  a  discounted  cash  flow  model 
for  the  payments  and  costs  and  applying  penalties  to  the  payments  when  the  schedule  is 
not  met.  These  methods  are  the  subject  of  current  work. 


Simulation  Implementation 

A  simulation  model  using  the  preceding  constructs  has  been  implemented  using  an 
organizational  simulation  framework.  This  framework  uses  AnyLogic,  a  commercially 
available  simulation  software  product,  plus  a  set  of  customized  classes  to  model 
organizational  artifacts  and  behaviors  (Bodner,  2009).  This  allows  actors  to  be  modeled 
using  an  agent-based  approach  and  acquisition  processes  to  be  modeled  using  a  discrete- 
event,  process/transactional  approach.  The  interaction  of  these  two  approaches  is  shown  in 
Figure  3.  Note  that  not  all  possible  behavioral  interactions  or  structural  relationships  are 
shown. 


•  •  • 


Activity  n 


Figure  3.  Process-Actor  Interaction 

This  paper  focuses  on  the  implementation  of  a  principal-agent  model  and  an 
associated  cost  model  within  one  acquisition  phase.  Each  year  within  the  phase,  the 
following  occur: 

■  The  agent  determines  its  level  of  effort. 

■  Based  on  the  level  of  effort  and  the  probabilities,  a  value  for  the  performance 
level  is  computed. 

■  The  principal’s  cost  incurred  is  updated  with  the  relevant  cost  from  the  year  in 
question. 

■  Based  on  the  cost  incurred,  cost  growth  is  measured. 
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■  The  estimated  time  to  completion  is  updated  based  on  the  performance  for 
the  year. 

■  The  estimated  cost  for  the  acquisition  phase  is  updated. 

■  Cost  growth  for  the  acquisition  phase  is  estimated. 

■  The  principal  and  agent  interact,  with  new  probabilities  being  generated. 

At  present,  we  assume  that  the  contract  structure  and  amount  are  not  renegotiated. 

Experimental  Example 

This  section  describes  example  simulation  results  from  the  cost-plus  incentive  model 
described  in  the  Model  Description  section.  The  model  features  a  principal  and  an  agent 
that  interact  annually  over  a  particular  acquisition  phase.  At  each  interaction,  the 
probabilities  for  agent  performance  change.  Cost  accrual  is  tracked,  as  is  time  taken  to 
complete  the  phase  by  the  agent. 

Parameters 

Tables  1-4  contain  parameters  used  in  the  simulation  example. 


Table  1.  Probability  of  Performance 


Low  Effort 

High  Effort 

Low 

Medium 

High 

Low 

Medium 

High 

0.6 

0.3 

0.1 

0.1 

0.3 

0.6 

Table  2.  Principal’s  Cost  Based  on  Performance 


Low 

Median 

High 

100 

200 

400 

Table  3.  Agent’s  Cost 


Low  effort 

High  Effort 

50 

150 

Table  4.  Other  Parameters 


Est.  Phase  Duration  (Yrs.) 

Interaction  Frequency 

Reserve  Utility 

10 

Annually 

100 
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Probabilistic  Performance 


To  model  changes  in  the  probabilities  (pf)  associated  with  achieving  a  certain 
performance  level  given  a  level  of  effort,  we  use  two  methods. 


■  Random  assignment.  Each  p'is  assigned  a  new  value  from  the  Uniform  (0, 
1)  distribution,  subject  to  the  earlier  constraints  (j?j  >  and 

EWSEB- 


Random  addition.  Each  p^is  assigned  a  new  value  by  adding  a  random 
amount  to  the  previous  value.  The  new  value  is  p  (l  f , 

where  r=  0.1 .  This  simulates  a  random  walk  process.  The  same  constraints 
are  observed,  as  well  as  the  constraint  that 


Example  Results 

Using  the  random  assignment  of  probabilities,  the  following  results  are  obtained,  as 
shown  in  Figure  4.  The  initial  phase  duration  estimate  of  ten  years  was  proven  to  be 
incorrect,  since  the  phase  lasted  fifteen  years.  The  estimated  cost  remained  relatively 
constant,  and  the  final  cost  ($3,800)  was  within  a  95%  confidence  interval  for  the  mean  of 
the  estimated  total  costs  from  each  year  ($3,727,  $3,982). 


Figure  4.  Cost  Accrual  Using  Random  Assignment 

Under  random  addition,  the  following  results  are  obtained,  as  shown  in  Figure  5. 
Similar  to  the  previous  example,  the  initial  estimate  for  phase  duration  was  incorrect,  with 
the  actual  value  being  twelve  years.  The  actual  cost  of  $3,600  was  likewise  within  a  95% 
confidence  interval  for  the  mean  of  the  estimated  total  costs  from  each  year  ($3,425, 
$3,953). 
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Figure  5.  Cost  Accrual  Using  Random  Addition 

In  the  second  case,  there  is  less  variation  in  the  probability  values  across  time  as 
compared  to  the  first  case.  This  is  due  to  the  probability  update  mechanism  of  the  second 
(random  addition),  which  changes  the  probabilities  within  a  smaller  range  than  the  first.  This 
is  due  to  the  relatively  small  value  of  r.  Thus,  in  the  first  case,  the  agent  can  switch  much 
more  readily  between  low  effort  and  high  effort,  while  in  the  second  case,  the  switch  occurs 
less  often.  Thus,  we  see  that  this  type  of  variability  can  drive  longer  phase  durations. 

In  both  examples,  there  is  little  to  no  cost  growth.  This  occurs  largely  due  to  three 
reasons.  First,  the  payment  differential  between  high  performance  and  low  performance  is 
relatively  large.  This  motivates  the  agent  to  expend  high  effort.  A  smaller  differential  would 
likely  cause  increased  expenditure  of  low  effort  (at  relatively  higher  cost),  resulting  in  cost 
growth.  Second,  the  example  does  not  use  an  initial  cost  estimate  derived  independently 
from  the  estimate  used  during  program  performance.  Such  independent  estimates  may  be 
artificially  low.  Finally,  technical  issues  are  not  modeled  that  would  tend  to  drive  cost  growth 
(e.g.,  immature  technologies,  production  quality  issues).  These  issues  have  been 
demonstrated  in  previous  work  (Bodner  et  al.,  2010). 

Discussion  and  Future  Research 

This  paper  has  presented  a  model  for  the  acquisition  enterprise  that  addresses  its 
process-oriented  aspects,  as  well  as  incentives  for  the  actors  involved.  The  incentive  model 
is  derived  from  the  principal-agent  model  used  in  micro-economics.  The  acquisition  model 
has  been  implemented  using  an  organizational  simulation  framework.  Sample  results  are 
presented  to  demonstrate  the  behavior  of  a  government  principal  and  a  prime  contractor 
agent. 

Current  work  addresses  the  scale-up  of  the  model  to  more  realistic  acquisition 
situations  and  the  integration  of  more  detailed  process  models  (which  cover  the  multi-phase 
acquisition  life  cycle  and  associated  technical  drivers  of  cost  growth)  with  the  actor  models. 
Of  primary  interest  are  programs  such  as  the  F-35  Joint  Strike  Fighter.  In  this  program,  cost 
growth  has  been  an  issue,  due  to  technical  process  issues  and  due  to  incentives.  In 
addition,  the  F-35  employs  a  systems  integrator  paradigm  of  acquisition  in  which  the  prime 
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contractor  integrates  complex  subsystems  designed  and  produced  by  a  network  of  other 
contractors.  This  paradigm  is  relatively  new,  since  it  was  preceded  for  many  years  by  the 
manufacturer  paradigm,  in  which  most  design  and  production  were  performed  by  the  prime 
contractor,  with  relatively  simple  subsystems  contracted  out  according  to  the  prime’s 
specifications.  It  is  increasingly  being  used  in  acquisition  of  complex  systems  by  the  DoD 
and  the  industry  (e.g.,  Boeing  Dreamliner).  The  systems  integrator  paradigm  implies  the 
need  for  extensive  research  in  incentives  and  contract  structures.  Due  to  the  complexity  of 
supply  networks,  simulation  will  be  a  valuable  tool  in  this  type  of  research. 
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Motivation 
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Cost  growth  evident  in 
weapons  programs 

-  $296  billion  in  2008 
portfolio 

-  $135  billion  since  2008 

-  $70  billion  unexplained  by 

uantit  chan  es 

I  y  w 

Pressure  to  rein  in  costs  due  to 
fiscal  and  political  environment 

Decision-making  regarding 
processes  and  incentives 


Process  Drivers 
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•  Evolutionar  ac  uisition 

*  i 

-  Each  development  cycle  occurs  at  lower  cost 

-  Increased  number  of  development  cycles  contributes  to 
potentially  higher  cost 

•  Phase  concurrency 

-  Concurrency  might  be  used  to  regain  schedule  (e.g.,  between 
development  and  production) 

-  Concurrency  introduces  risk  of  rework  and  wasted  production 

•  n  rt  int^ 

-  Technology  immaturity  poses  cost  growth  risk 

-  Requirements  volatility  poses  cost  growth  risk 
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Incentive  Drivers 
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•  Cost-plus  contracts 

-  Cost  growth  tends  to  be  enabled 

•  Competition  vs.  non-competition 

-  Competition  should  incentivize  cost  performance 

-  Low  bids  are  potentially  incentivized  by  competition 

•  Incentives 

-  Evidence  shows  award/incentive  fees  ineffective 

-  Recommendations  for  using  fees  include 

•  Set  base  fee  and  tie  overall  fee  to  outcomes,  not  time 

•  Using  rollovers  judiciously 
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Organizational  Simulation  Ge  T^lh  | 


Simulation  methodology 
focused  on  the  organizational 
experience 

-  People 

-  Social  behavior 

-  Rules  and  processes 

-  Artifacts 

-  Architecture 

Tools  for  designing,  testing, 
prototyping  and  experimenting 
with  organizational  systems 

AnyLogic  implementation  with 
Java  class  library  for 
organizational  modeling 
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Process  Model 


AAA 


Concept 

Technology 

System  Dev.  & 

Production 

Operations 

Refinement 

Dev. 

Demonstration 

& 

&  Support 

Deployment 
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Possible  contract  renegotiations 
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•  Ac  uisition  hases 

i  i 

•  Decision  points 

•  Cost  accruals 

•  Progress 

-  Technology  maturation 

-  Design 

-  Development 

•  Concurrency  and 
uncertainty 

•  Contract  renegotiations 
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Actor  Model 


G  =  Government 
P  =  Prime  contractor 
C  =  Contractor 
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•  Principal-agent  model 

-  Government  as  principal 

-  Contractor  as  agent 

•  Are  tne  interests  ot  tne 
agent  aligned  with  those 
of  the  principal? 

-  Contract  structure 

•  Eventually  extended  to 
multi-tier  principal-agent 
network 

-  Complex 

-  Non-transparent 
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Incentive  Model 
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•  A  ent  has  a  utilit  U(w,  a  from  workin  for  the  rinci  al 

w  ^  \  1  /  w  i  i 

-  w  =  payment 
-a  -  effort 

-  Reserve  utility  in  case  of  no  contract 

•  Payment  each  period 

-  x02  for  low  performance 

-  xf  for  medium  performance 

-  x22  for  high  performance 

•  Performance  based  on  effort,  incorporatiny  unueiicuniy 

-  P(a/)  =  Pj  with  probability  pj 

-  Probabilities  scaled  so  that  lower  efforts  have  higher  probabilities 
of  low  performance  and  vice-versa 
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Optimization 


Georgia  II  TTcMoo’ocMraiMMDifD 
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•  Princi  al’s  ers  ective 

i  ii 

•  Minimize  expected  payout 

•  Subject  to 

-  Agent’s  expected  utility  >  reservation  utility 

-  Agent’s  expected  utility  for  high  effort  >  expected  utility  for  low 
effort 
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Cost-Plus 


•  Let  T  be  total  time  for  an  acquisition  phase,  and  let  there  be  an 
estimate  of  time  remaining  at  time  t 

•  Agent’s  progress  each  period  goes  toward  completion  time,  and 
estimate  of  time  remaining  is  updated 

•  Principal  pays  x,2  per  period  depending  on  performance  level 

•  An  initial  cost  estimate  is  provided,  and  is  updated  with  actual  cost 
incurred  lus  an  estimate  for  the  remainin  hase  duration 

I  W  I 

•  Cost  growth  and  incremental  cost  growth  can  be  measured  by  the 
difference  of  the  actual  and  the  estimate  at  time  t 
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Fixed  Price 
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lech  U 


•  Assume  Cobb-Dou  las  roduction  function  with 

w  i 

increasing  returns  to  scale 

•  Production  efficiency  is  a  function  of  effort,  incorporating 
uncertainty 

-  a(a,)  =  ccj  with  probability  pj 

•  Agent  selects  level  of  effort  to  maximize  pected  gain 

-  Price  per  unit  multiplied  by  number  of  units  less  expected  cost  to 
agent  based  on  production  function 

•  Discussion 

-  Production  efficiency  of  concern  primarily  to  agent 

-  Principal  concerned  with  schedule 

-  Discounted  cash  flows  and  penalties 
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Simulation  Implementation  Gei«ch 


•  Agent  determines 
effort 


•  Performance  level 
computed 

•  Cost  incurred  updated 

•  Estimated  time  to 
com  letion  u  dated 

i  i 

•  Estimated  cost 
updated 

•  Cost  growth 
estimated 


\  ' 
\  \ 
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•  New  probabilities 
generated 
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Probabilities 
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•  Probabilities  chan  e  at  each  eriod 

w  i 

•  Random  assignment 

-  Each  probability  is  assigned  a  new  value  from  the  Uniform  (0, 1 ) 
distribution,  subject  to  the  earlier  constraints 

•  Random  addition 

-  Each  robabilit  is  assi  ned  a  new  value  b  addin  a  random 
amount  to  the  previous  value 

-  The  new  value  is  the  probability  multiplied  by  (1  +rUnif(-1 ,1 )), 
where  r  =  0.1 

-  This  simulates  a  random  walk  process 

-  The  same  constraints  are  observed,  as  well  as  the  constraint 
that  probabilities  are  non-negative 
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Probability  of  performance 


Low  Effort 

High  Effort 

Low 

Medium 

High 

Low 

Medium 

High 

0.6 

0.3 

0.1 

0.1 

0.3 

0.6 

Principal’s  cost  based  on  performance 


Low 

Median 

High 

100 

200 

400 

Agent’s  cost 


Low  effort 

High  Effort 

50 

150 

Other  parameters 


Est.  Phase 
Duration  (Yrs.) 

Interaction 

Freguency 

Reserve  Utility 

10 

Annually 

100 
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Random  Assignment  Ge  T^lh  | 


Estiamted  total 
Cost 


Actual  Cost 
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Estiamted  total  Cost 

►Actual  Cost 
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Equal  Probabilities 


Random  Assi< 

gnment 

Random  Addition 

Original 

Equal  Prob 

Original 

Equal  Prob 

Mean 

177.3 

185.3 

169.5 

175.9 

Std.  Dev. 

61.2 

60.7 

76.0 

47.2 

•  Random  addition  has  a  lower  variance  than  random 
assi  nment  due  to  the  enerall  smaller  chan  es  in 
probabilities  between  periods 

•  This  difference  is  smaller  under  the  equal  initial 
probabilities  since  there  is  less  switching  between  effort 
levels 
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Discussion 
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•  Combined  rocess  and  incentive  modelin 

i  w 

•  Micro-economic  and  technical  behavior  models 

•  Contract  structure 

•  Simulation  needed  due  to  complexity 


Knowledge  and  Skills  for  Enterprise  Transformation. 


19 


Current  Research 
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•  Scale  up  work  with  F-35  data  and  multi-tier .  rinci.  al- 
agent  network 

•  Enhance  fixed  cost  model 

•  Experimentation  with  incentive  structures,  transfer  points 
and  process  concurrency 
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